OUTER AUTOMORPHISMS OF MAPPING CLASS 
GROUPS OF NONORIENTABLE SURFACES 
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Abstract. 

genus g > 5 and Mod(Ai'g) denote the mapping class group of Ng. We 
prove that the outer automorphism group of Mod(7Vg) is either trivial 
or Z if g is odd, and injects into the mapping class group of sphere with 
four holes if g is even. 



1. Introduction and statement of results 

Let Ng^n be a connected nonorientable surface of genus g with n holes 
(boundary components) and let Mod{Ng^n) denote the mapping class group 
of Ng^n, the group of isotopy classes of all diffeomorphisms Ng^n ^g,n- 
Our work is motivated by the following theorem of N. V. Ivanov. 

Theorem 1.1. (Ivanov) Let M = Mod(5), PMod(5), Mod*(5) or PMod*(5). 
If S is neither a sphere with at most 4 holes, nor a torus with at most 2 
holes, nor a closed surface of genus 2, then every automorphism of M is 
given by the restriction of an inner automorphism of Mod* (S). In partic- 
ular, Out(Mod(5)) is a finite group and moreover, Out (Mod*(5')) = 1, 
Out(Mod(5)) = Z/2Z. 

In the next section we study the orientation double cover of a nonori- 
entable surface and we investigate relation between mapping class groups of 
the nonorientable surface and its orientation double cover. Furthermore, we 
describe maximal abelian subgroups of the mapping class group of nonori- 
entable surfaces and construct some special type of abelian subgroup of the 
mapping class group of nonorientable surface. 

In Section [3] we characterize the Dehn twists about circles on a closed 
nonorientable surface using the ideas of Ivanov's analogous work on the 
orientable surface 

In Sectional we prove that the outer automorphism of Mod{N2g+i) injects 
into Z if > 2, on the other hand, the outer automorphism of Mod(A''2g) 
injects into the mapping class group of sphere with 4 holes if 5 > 3. 
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Finally, we compute the outer automorphism of mapping class groups of 
some sporadic cases using the presentations of mapping class groups of some 
nonorientable surfaces not covered in our theorems. 

2. Preliminaries and Notations 

Let ^g,n and Ng^n denote the connected orientable surface of genus g 
with n holes and the connected nonorientable surface of genus g with n 
holes, respectively. Recall that the genus of a nonorientable surface is the 
maximum number of projective planes in a connected sum decomposition. 

2.1. The orientation double cover of Ng^n- Let p : 2n ^g,n be 

the orientation double cover of the nonorientable surface Ng^n with the Deck 
transformation a : Sg_i^2n ^g-i,2n- Then p*(7ri(Sg_i^2n) a^)) is a normal 
subgroup of index 2 in 7ri(Ng^n,pix)) and the deck transformation group is 
{a \ = identity) ^ Z2. If F' is a diffeomorphism of S„_i_2n such that 
a o F' = F' o a, then F' induces a diffeomorphism F on Ng^n- 

Let L»i//(S^_i,2n)<"> = {F' £ Z)i//(S3_i,2„,) \ F' = a o F' o a'^}. So, 
we have a homomorphism Dif f{Yig^i^2n)^'^^ F)if f{Ng^n)- Moreover, this 
homomorphism is onto. Indeed, we have an exact sequence 

^ Z2 ^ Z)i//(S3_i,2„)<"> ^ Diff{Ng^n) ^ 0. 

On the other hand, any diffeomorphism F of the nonorientable surface Ng^n 
has a unique lift to an orientation preserving diffeomorphism F' : Eg_i^2n — > 
Sg_i^2n- Hence, the above exact sequence splits and therefore we can iden- 
tify Dif f{Ng^n) with the subgroup Di//+(Sg_i^2n)^'^'* , the subgroup of ori- 
entation preserving diffeomorphisms of Sg_i^2n commuting with the deck 
transformation a. 

Indeed, any isotopy of the nonorientable surface Ng^n lifts to the sur- 
face Sg_i^2n- Moreover, if the Euler characteristic of the surface is negative 
(hence it carries a complete hyperbolic metric) then any isotopy of orien- 
tation preserving diffeomorphisms of T,g-i^2n, whose end maps commute 
with the deck transformation a, can be deformed into one which commutes 
with a, and hence descends to an isotopy of Dif f{Ng^n) (cf. Lemma 5 in 
[8]). Therefore, we can identify the mapping class group Mod(A'g^„) as the 
subgroup Mod(Sg_i^2n)^'^\ the subgroup of orientation preserving mapping 
classes of Sg_i^2n commuting with the mapping class of the deck transfor- 
mation a. 

2.2. Circles. If a is a circle on Ng^n, by which we mean a simple closed 
curve, then according to whether a regular neighborhood of a is an annulus 
or a Mobius strip, we call a two-sided or one-sided simple closed curve, 
respectively. 

We say that a circle is nontrivial if it bounds neither a disc with at 
most one puncture nor annulus together with a boundary component, nor a 
Mobius band on Ng^n- 
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If a is a circle, then we denote the complement of a by Ng .^ the surface 
obtained by cutting Ng^n along a. A circle a is called nonseparating if iV^ „ 
is connected and otherwise, it is called separating. 

Following Ivanov [T], we will call a one dimensional submanifold C of a 
surface S a system of circles if C consists of several pairwise nonisotopic 
circles. By S'^ we will denote the surface S cut along all the circles in the 
system C. 

Definition 2.1. A finite sequence of embedded two-sided circles {ci, • • • , c^} 
on the any surface is called a chain if the geometric intersection number of 
the isotopy classes 7^ and 7^, satisfies i(7j,7j) = ^i+ij, for all where 
the geometric intersection number defined to be the infimum of the 

cardinality of di H dj with di G 'ji = [cj], dj € 7^ = [cj]. 

2.3. Periodic, reducible, and pseudo-Anosov elements. For an ori- 

entable surface we have the classical definition below. 

Definition 2.2. Let F : '^g,n be any diffeomorphism. 

• -F is called periodic if it is isotopic to some diffeomorphism F with 
F^ = id for some k 0, 

• F is called reducible if there is a system of circles C 7^ on and 
a diffeomorphism F, isotopic to F, with F{C) = C, 

• otherwise, F is called pseudo-Anosov. 

If F is periodic, or reducible or pseudo-Anosov, its isotopy class / is called 
periodic element, reducible element, pseudo-Anosov element, respectively. 

It has been shown in [8] that one can define periodic, reducible and 
pseudo-Anosov diffeomorphisms for nonorientable surfaces exactly in the 
same way. Moreover, if F is a diffeomorphism of the nonorientable surface 
Ng^n with its unique orientation preserving lift F' to the orientation double 
cover Eg_i_2n) then F is periodic, reducible or pseudo-Anosov if and only if 
F' is periodic, reducible or pseudo-Anosov, respectively. 

2.4. Pure elements. 

Definition 2.3. A diffeomorphism F : Ng^n ^g,n is called pure if for 
some system of circles C the following condition is satisfied: All points of C 
and dNg^n are fixed by F, F does not interchange components of Ng^^ and 
it induces on every component of -/V^„ a diffeomorphism, which is either 
isotopic to a pseudo-Anosov one, or to the identity. 

2.5. Abelian subgroups of r'(m). Let r(m), where m G Z, m > 1, be 
the kernel of the natural homomorphism 

Mod(S<,_i,2„) ^ Aut(ii'i(S3_i,2n,Z/mZ)). 

Then T{m) is a subgroup of finite index in Mod(5]g_i.2n)- Let T'{m) = 
r(m) n Mod(A''g_„), regarding yioA{Ng^n) as a subgroup of Mod(Sg_i^2n) as 
described above. 
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It is well known that T{m) and hence T'{m) consist of pure elements only 
provided that m > 2, ([Ij). Throughout the paper, we will consider T(m) 
for only m > 2. 

Following Ivanov, suppose that C is a system of circles on Ng^n and a dif- 
feomorphism Fp : P — > P is fixed on each component P of N^^. Moreover, 
suppose that Fp is fixed at all points of the boundary of P and is either 
isotopic to the identity or to a pseudo-Anosov map on P. We extend Fp 
to Fp : Ng^n ^g,n by the identity. Let the subgroup A of Mod(A'g^„) be 
generated by /p's, which are the isotopy classes of such Pp's, and all Dehn 
twists about two-sided circles in C. The subgroup A is abelian. Indeed, in the 
orientable surface case, Ivanov showed that any maximal abelian subgroup 
of r(m) is of this type (^J). The analogous result holds in the nonorientable 
case also: 

Lemma 2.4. Every abelian subgroup of T'{m) is contained in some abelian 
subgroup constructed as in the above paragraph. 

Proof. Consider an abelian subgroup K < T'{m). Regarding K < T{m) let 
C be a reduction system for K described by Ivanov for orientable surfaces [1]. 
For any x £ C C C and f G K, we have f{a{x)) = a{f{x)) = o"(x), because 
each f G K \s invariant under the involution a : Sg-i,2n '^g-i,2n- So we 
have (j{C) = C. Let C € C be a circle in the system with ct(C) = C. Since 
a has no fixed points, the restriction of a to the circle C is an orientation 
preserving involution on that circle. So each circle of the system C maps 
onto a circle in Ng^n and hence the system C gives a system C' on Ng^n- 
Now X C A, where A is an abelian subgroup corresponding to the system 
of circles C, as described in the above paragraph. □ 

The system of circles C constructed in the above proof will be called a 
reduction system for the abelian subgroup K. 

Let r be a subgroup of finite index of T{m). Then, the its a invariant 
part, r' = {a; G r I axa~^ = x}, is also finite index subgroup of r'(m). 

Lemma 2.5. The center of every subgroup of finite index of T'{m) is trivial. 

Proof. The centralizer in r(m) of a pseudo-Anosov element is isomorphic to 
Z, ([I]). Therefore, the centralizer in T'{m) of a pseudo-Anosov element is 
also isomorphic to Z. From this, we obtain that the center of every subgroup 
of finite index of T'(m) is trivial. □ 

Stukow's result [7J on Dehn twists on nonorientable surfaces shows that 
Dehn twists on nonorientable surfaces enjoy the same properties with those 
on orientable surfaces. For instance, two Dehn twists commute if and only 
if their circles can be chosen disjoint; they satisfy the braid relation if and 
only if their circles meet transversally at one point. 

Before we go further we give the following remark. 

Remark 2.6. a) The pure mapping class group of real projective plane 
with less than or equal to two holes is finite. 
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b) The pure mapping class group of the Klein bottle is finite and that of 
the Klein bottle with one hole has no pseudo-Anosov element (see Proposi- 
tion IM]). 

c) The pure mapping class group of Klein bottle with two holes contains 
Z0Z. 

d) The pure mapping class group of N^^n {n > 0) contains a copy of Z Z, 
though that of N^, does not support any pseudo-Anosov diffeomorphism ([4J). 

Using the above remark we deduce that the smallest surfaces which sup- 
port a pseudo-Anosov diffeomorphism in their pure mapping class group 
which does not have an abelian subgroup of rank two are 5]o,4, Si, Si^i or 
A^i^3. Examples of pseudo-Anosov mappings on the surfaces So,4) and 
Si^i are well known. For the nonorientable surface the map tatbtc, 

described in the Figure [U is indeed pseudo-Anosov (Theorem 4.1 in [3]). 




Figure 1. The real projective plane with three boundary components 

We will finish this section with the following result, whose proof will be 
omitted. 

Lemma 2.7. Let K < T'{m) he a maximal abelian subgroup of Ng of highest 
rank, where g is at least three. Then the rank (K) = ^^^^ — 3 and each 
component of Ng is either sphere with three holes, sphere with four holes, 
torus with one hole, or real projective plane with two or three holes, where C 
is a minimal reduction system for K and r is 1 or depending on whether 
g is odd or even. 

Moreover, in both even and odd genus cases, one can choose maximal 
abelian subgroups of highest rank so that they are generated by Dehn twists 
about nonseparating circles. 

3. Characterization of Dehn twists 

3.1. Algebraic characterization of Dehn twists. Throughout this sec- 
tion r' is a finite index subgroup of T'{m), m > 2. 
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Theorem 3.1. Let Ng be a compact closed connected nonorientable surface 
of odd (respectively, even) genus g > 5 (respectively, g > 6). An element 
f € Mod(Ng) is a Dehn twist about a nonseparating circle (respectively, 
nonseparating circle whose complement is nonorientable) if and only if the 
following conditions are satisfied: 

i) C(Cr'(/")) is isomorphic to TL, for any integer n ^ such that 

/" G r. 

ii) There is an abelian subgroup K of rank -^-^ — (respectively, -^-^ — ) 
generated by f and its conjugates freely. 

iii) / is a primitive element of Cy[od{Ng){K). 

Proof. We will prove the odd genus case mainly following the proof of Ivanov 
in [1]: The even genus case is basically the same with minor changes, which 
we will emphasize in the when needed. 

If the above conditions are satisfied, then we need to show that / is a 
Dehn twist about a nonseparating circle. 

1 1 1 % — 7 ,3g — 8 . , . , 

Let us denote the number — ^ — ( — ^ — , m the even genus case) by s. 

If is equal to the identity, then Cr'{f^) = T' and hence by the first 
condition, C(r') is isomorphic to Z. However, this is a contradiction by 
Lemma 12.51 Therefore, /" is not equal to the identity. 

We will denote a minimal reduction system of /" by C , and the subgroup 
generated by twists about two-sided circles in C' by G. Set G' = G HT' . G 
and G' are free abelian groups. First we will show that G' C C(Cr'(/"')). 
Let g G Gr'if"'). This implies that /" and g commute and hence /" and g 
belong to some abelian subgroup A constructed as in the previous section. 
Let C denote the system of circles used in the construction of A. Therefore, 
C is isotopic to a subsystem of C, because /" G A. Thus, we have G C A. So, 
each element of G commutes with all 5 G A. Therefore, we obtain that G C 
CMod(iv,)(Cr'(/")) and we obtain that G' C Cr'(Cr'(/")) C C(C7r. (/"))• 

By the assumption that C{Cr'{f^'')) = we have C has no or just 
one two-sided circle. Suppose that C' has no two-sided circle. Say C = 
{ci, • • • ,Cfc}, where Cj is a one-sided circle for all i. Then Ng is connected 
and the restriction /, ^c' is either the identity or a pseudo-Anosov. 

If /|^c' is the identity, then / must be a product of Dehn twists about 

some circles in C which is not possible since each Cj is one-sided. Therefore, 
/|^c' is a pseudo-Anosov. However, the maximal abelian group containing / 
has rank one. By the assumption g > 5 and thus the rank is s > 4 > 1. This 
is a contradiction. (Similarly, in the even genus case, we have s > 5 > 1.) 
So, C has exactly one two-sided circle and so C = {ci, • • • , Cfc, a}, where a 
is a two-sided circle and each Cj is one-sided. 

Let D be the group generated by /" and the twist about a and set D' = 
D n r'. Then, we have D' C G{Cr'{f"')) and hence D' is isomorphic to Z. 
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It follows that there are no pseudo-Anosov diffeomorphisms in the Thurston 
normal form of and thus is a power of the Dehn twist ta about a. So, 

= for some integer m. 

Now we need to show that a is nonseparating. We assume that a is a 
separating circle. So, A''^ = NiU N2 such that Xj = xi^j) ^^'^ Xi + X2 = 
x{Ng). Since a is nontrivial we have < for j = 1, 2. Without loss of 
generality, assume xi ^ X2- 

Let /i, • • • , /s be the elements conjugate to / generating the abelian sub- 
group K in the statement of the theorem. Then /" = t^, where a, is a 
circle. Similarly, ai separates the surface into two components, say N\ and 
which are diffcomorphic to Ni and N2. respectively. By the structure 
of abelian groups, the circles can be chosen to be disjoint and pairwise 
nonisotopic. Moreover, 7V| is diffeomorphic to Nl and ai is not isotopic 
to aj for any i / j. So, Nl cannot be contained inside Nl. Therefore, if 
Ni n n{ / 0, then N\ U n{ = Ng. 

Since Oj is not isotopic to aj, we see that xi^l ^ N^) < 0. Hence, we see 
thatx(iv,) = xiNi)+xiNi)-x{NinNi) = xi+xi-xWniVf) > xi+Xi > 
Xi + X2 = x(-Ag) which is a contradiction. So, NlnN^ = 0, in other words, 
the surfaces N^ arc all disjoint. Let A" = Cl{Ng \ {N^ U • • • U A^f )). The fact 
that flj arc being pairwise nonisotopic implies that A^*^ cannot an annulus. 
Also, A^^ is not a sphere, a torus, a Klein bottle, a projective plane, a disk 
or a Mobius band. Thus, x(-^°) < 0. 

Now, 2-g = x{Ng) = x(A^°) + x(A^i') + • • • + x(A^f )• So, 2 - g < sx{Nl). 
Then, g — 2 > s \ xi^i) I- So we get 2g — 4 > 3g — 7, which implies 3 > g. 
This is a contradiction. Thus, a is a nonseparating circle. (For even genus, 
we would have 2^ — 4 > 3^ — 8, which would yield 4 > 5, a contradiction to 
the assumption that g > 6.) 

We take an element € C'Mod(Afg)(-^)- ^ • -^9 ^ -^5 denotes a diffeomor- 
phism in the isotopy class ^. The group K is generated by /i, • • • , fg and 
fi = C ^ before. Since G CMod(Ng){K), tpfiip~^ = h and thus V'/f = 

. In other words, V'^^'V'"^ = and so ) = ■ So, replacing by 
an isotopy we can assume that ^{ai) = a^. Let C = {ai} U • • • U {a^}. 

Since s is the maximal number of pairwise nonisotopic disjoint two-sided 
circles on Ng, every component of Ng is a disc with two holes or a one holed 
Mobius band. Total number of the boundary components of the components 
of Ng is 2s and hence all the components of Ng except one must be pair of 
pants. (In the even genus case, s + 1 is the maximal number of nonisotopic 
two-sided circles on Ng. Now, similar arguments prove that all the compo- 
nents A^ but one are discs with two holes, and the remaining component is 
a sphere with four holes.) 

Suppose that ^ sends one of these components to another one. Since 
* fixes the circles(boundary components) of Ng, we see that Ng = P U R 
where *(-P) = R. This is a contradiction to the assumption that Ng is a 
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nonorientable surface of odd genus. So ^ does not permute the components. 
Hence, ^ is a composition of twists about m. (Again, in the even genus case, 
since there is only one sphere with four holes component of Ng, we see that 
^ cannot interchange the components.) 

Now note that, / G CMod{Ng)iK) because f G K, and K is abelian. 
Without loss of generality, suppose that f = fi- So, = fi = t^^- So by 
the above paragraph, we obtain that / is a power of ta^; f = t^^. for some 
/, because K is free abelian. Therefore, fi = t^^ and K is generated by t\^. 
Thus, ta is in Cyiod{N By the last condition, / is primitive and hence, 

f = ta. 

In the even genus case, one has to prove that the complement of a is 
nonorientable. Assume that the complement of a is orientable. Since each 
ta- is conjugate to ta the complement of each is also orientable. It is easy 
to see that each is a Z2 homology cycle Poincare dual to the first Stiefel- 
Whitney class wi of the surface Ng. In particular, for any 1 < ^, j < s, 
i / j, ai and aj form a boundary of a subsurface Sjj of Ng. Moreover, 
since the complement of each Oj is orientable, S'jj- is an orientable surface 
of genus at least one with two boundary components. By relabeling Oj's we 
may assume that the interior of each Sj^j+i (where we set Sg^s+i = Ss^i) does 
not intersect any a^. Hence, n •S'jj+i = 0, if and only if \i — j\ > 1. 

Now, Ng = Ui>i>sSi^i+i, and therefore we get g>2s + 2 = 3g — 8 + 2 = 
3g — 6 > g, a contradiction since 5 > 6. Hence, the complement of a must 
be nonorientable. 

The other direction of the theorem is straight forward and left to the 
reader (see also [1]). □ 

Similar arguments prove the following theorems (compare with the cor- 
responding result in [1]): 

Theorem 3.2. Let g > 5 be an odd integer. An element f € Mod(A'p) is a 
Dehn twist if and only if the following conditions are satisfied: 

i) C{CY'{f'^)) is isomorphic to 7L, for any integer n ^ such that 

/" G r. 

ii) There exists a free abelian subgroup K of Mod(A'g) generated by f 

3g — 9 

and — ^ — twists about two-sided nonseparating circles such that 

rank (K) = -^-^ — . 

iii) / is a primitive element of Cy[od{Ng){K). 

For Dehn twists on even genus surfaces we have, 

Theorem 3.3. Let g > 6 be an even integer. An element f € Mod{Ng) is 
a Dehn twist if and only if the following conditions are satisfied: 

i) C{CT'{f^)) is isomorphic to Z, for any integer n ^ such that 
/" G F'. 
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ii) There exists a free ahelian subgroup K of Mod(A'g) generated by f 
and — ~ Dehn twists about nonseparating circles whose comple- 
ments are nonorientable, such that rank (K) = -^—^ — . 

iii) / is a primitive element of Cyiod(Ng)[K)- 

Remark 3.4. 1) Let <I> : Mod(A^) — > Mod(A^) be an automorphism, where 
the genus of N is at least five. Since V is a subgroup of T'[m) of finite 
index the subgroup <I>(r') n T'[m) is stiU a finite index subgroup of T'{m). 
Hence the above theorems give an algebraic characterization of a Dehn twist. 
Therefore, we can also characterize algebraically the subgroup T of Mod(A^) 
generated by all Dehn twists. 

2) A two-sided embedded circle on a closed nonorientable surface of odd 
genus is either separating or nonseparating with nonorientable complement. 
Therefore, by Theorem 13.11 and Theorem 13.21 an automorphism of the map- 
ping class group respects the topological type of the circles of the Dehn 
twists. 

However, in case of nonorientable surfaces of even genus. Theorem 13 . 1 1 and 
Theorem 13.31 are not strong enough to distinguish a Dehn twist about a sep- 
arating circle from a Dehn twist about a nonseparating circle with orientable 
complement. To make sure that automorphisms respect the topological type 
of the circles the Dehn twists are about, we need to study chains of Dehn 
twists. 

3.2. Chains of Dehn twists. Now we would like to discuss an application 
of the above theorems to the chains of Dehn twists, which we will use in the 
study outer automorphisms of mapping class groups of nonorientable sur- 
faces. Whenever we consider a chain or a tree of Dehn twists we will choose 
the circles the Dehn twists are about so that they intersect minimally (e.g. 
if the surface has negative Euler characteristic then we put an hyperbolic 
metric on it and choose the unique geodesies from each isotopy class). 

Stukow's results ([^) mentioned in Section 2 together with the above 
theorems imply that any chain of Dehn twists is mapped to a chain of Dehn 
twists under an automorphism of the mapping class group. 

On an orientable surface, a chain of circles is maximal if and only if it is 
separating. Moreover, in this case the chain has odd number of circles. This 
is no longer true on nonorientable surfaces of even genera. For example, a 
two sided circle with orientable complement forms a maximal chain of length 
one; however it is not separating. Nevertheless, we have the following result. 

Lemma 3.5. Let Ng be the nonorientable surface of genus g. 

i) If a chain of two sided nontrivial circles {ci,---q} on the closed 
surface Ng is maximal then I is an odd integer. Moreover, if g is 
an odd integer, then, the chain is maximal if and only if the chain 
separates the surface into two components. 
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In case of surfaces of even genus g, the chain {ci, • • • q} is max- 
imal if and only if it is ether separating, or nonseparating with ori- 
entahle complement. 
ii) Let {ci, • • • C2fc+i} he a separating maximal chain of two-sided non- 
trivial circles on Ng. Then it separates a disc if and only if there is 
no Dehn twist which commutes with all of the Dehn twists about the 
circles of the chain, but t^^ . 

Proof. The proof of the part (i) is left to the reader. 

For the statement in part (ii) , Figure O which estabhshes the proof in two 
special cases, indeed provides the proof of the "if part of the statement. 




(a) (b) 



Figure 2. 

For the other direction assume that there is a smooth curve C which 
intersects the chain transversally at finitely points of only ci and does not 
meet any other circle of the chain. Let S be any one of the two subsurfaces 
obtained by cutting the surface Ng along the chain. Since ci is the first 
element of the chain, the ci part of the boundary of S is connected. Note 
that 5 n C is a disjoint union of arcs. Now, if S" is a disc we can isotope 
these arcs outside the surface S so that C does not intersect the chain 
anymore. □ 

As mentioned in Remark 13.41 (1) we can use chains of Dehn twists to dis- 
tinguish algebraically Dehn twists about separating circles from Dehn twists 
about nonseparating circles with orientable complements on the nonori- 
entable surface of even genus A^^: Indeed, the longest chain on Ng, which 
lies in the complement of a nonseparating circle with orientable complement 
\s g — 1 (see also Lemma l3.8p . On the other hand, in the complement of a 
nontrivial separating circle the maximal length of a chain is again g — 1 but 
in this case the circle separates the surfaces into two components, a Klein 

bottle with one boundary component and an orientable surface of genus 
g-2 

— ^ — with one boundary component. Hence, in the case of nonseparating 

circle with orientable complement one can find one maximal chain of length 
5 — 1, whereas in the case of separating circles one can find two maximal 
chains, which lie in different components (and thus any element of one chain 
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commutes with any element of the other chain) of lengths g — 1 and 1. These 
arguments prove the following: 

Corollary 3.6. Let g > 6 be an even integer. An automorphism of the 
mapping class group of Ng maps a Dehn twist about a separating circle to a 
Dehn twist about a separating circle. Hence, an automorphism respects the 
topological types of the circles of the Dehn twists. 

We will end this section with a result which will be useful in the latter 
part of the paper. 

Corollary 3.7. Let g > 5 be an integer. An automorphism of the mapping 
class group of Ng maps any maximal chain of Dehn twists on Ng, which 
separates a disc, to a maximal chain of Dehn twists, which separates a disc. 

Proof. By Lemma 13.51 it is enough to prove that the image of a separating 
chain must be again separating. In the case of surfaces of odd genus a chain 
is separating if and only if it is maximal, which is a property preserved 
by algebraic automorphisms. So we may assume that {ci, ■ ■ ■ C2k+i} is a 
separating chain on the nonorientable surface Ng of even genus and the 
chain separates a disc from the surface. Hence, Ng cut along the chain is 
the disjoint union of a disc and a nonorientable surface with one boundary 
component of genus g — 2k, which is a positive even integer because g is 
even and the complement must be nonorientable. Therefore, there is an 
element, in the centralizer of the chain (i.e., it is a y homeomorphism which 
commutes with all elements of the chain) which is not a product of Dehn 
twists. 

Now assume that the image {di, ■ ■ ■ d2k+i}, of this chain under an auto- 
morphism of the mapping class group, is nonseparating. So by the above 

lemma, the complement of the chain is an orientable surface, say S, with two 

g-2k-2 

boundary components of genus . Now any mapping class, which 

is in the centralizer of the chain {di, ■ ■ ■ d2k+i}, has a representative F, a 
diffeomorphism of Ng, which maps a tubular neighborhood v, of the chain 
onto itself. So, F induces a diffeomorphism of both i/ and Ng — v. Since 
both pieces are orientable F is isotopic to a product of Dehn twists. How- 
ever, this contradicts to the fact that Dehn twists and hence their products 
are respected by automorphisms of the mapping class group. □ 

Indeed, similar arguments can used to prove that the topological type 
of other separating Dehn twists are preserved under automorphisms of the 
mapping class group. 

3.3. Characterization of Y-Homeomorphisms. As an application of 
the previous results we will an algebraic characterization of Y-homeomorphisms. 
By definition a Y-homeomorphism on a nonorienta,blG surface Ng^n 

is a dif- 
feomorphism whose isotopy class is not a product of Dehn twists but its 
square (composition with itself) is a Dehn twist about a separating circle 
which bounds a one holed Klein bottle (c.f. see [31 [U [Tj). 
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First wc give some easy observations: 

Lemma 3.8. The longest maximal chain in Mod(N2g+i) (respectively, in 
Mod(N2g)j has length 2g + 1 (respectively, 2g — 1). The longest chain 

in the centralizer of a Y -homeomorphism in Mod(N2g+i) (respectively, in 
Mod(N2g) ) has length 2g — 1 (respectively, 2g — 3). 

Proof. We will give the proof of the odd genus case only. The proof of the 
first statement is easy. For the second statement suppose that there is a 
maximal chain in the centralizer of a y-homeomorphism in Mod(N2g+i) of 
length of A; > 2g. Let ly he a tubular neighborhood of the chain of circles the 
Dehn twists of the chain are about. x{^) = xi'^kS^) = ^ — k. A maximal 
chain should be of odd length and thus k is odd. Then u has two boundary 
components. Since v is the orientable surface of genus (A: — l)/2 with two 
boundary components, '^(k-i)/2,2i 'we have xi^2g+i \ = 2 — {2g + 1) — 
{1 — k) = k — 2g > 1. This is a contradiction since N2g+i \ ^ niust contain a 
one holed Klein bottle. Hence, we get k>2g-\-\. Finally, it is easy to see 
that chains of claimed length exist in each case. □ 

Let r<Mod(Ng^n) denote the normal subgroup generated by Dehn twists. 

Theorem 3.9. An element f G Mod(N2g) is a Y -homeomorphism if and 
only if 

i) f^T, f^ = te, where te is a Dehn twist about separating circle e. 

ii) There is a chain of Dehn twist about nonseparating circles -Di, • ■ ■ , -D2g-i 
each of which has nonorientable complement, and such thatDi G Cy[od{N2g){^e), 
for all i^2g-2. Die CMod(N2g)(/); i<2g-S. 

Proof. The tubular neighborhood of the union of the circles corresponding 
to the chain Di, - ■ ■ , -D2g-3 is the orientable surface of genus g — 2 with 
two boundary components and hence, by assumption, / is supported in 
J^2g \ '^g-2,2, which is a copy of iV2,2 with boundary components, say ci and 

C2- 

Since e is separating in N2g and is nontrivial, e must separate a Klein 
bottle. Also D2g-\ and e do not meet but they are both contained in Ar2^2 
and hence we sec that i^2g-i is the unique two-sided circle in the Klein bottle 
bounded by e. By assumption the complement of -D2g-i is nonorientable and 
hence complement of Klein bottle must be nonorientable. Now, / must be 
a product of ) ^'^d some diffeomorphism supported in the Klein bottle 
bounded by e. However, = te implies that / has no and factors 
and hence must be supported in the Klein bottle bounded by e. Hence, it 
is a y-homeomorphism. 

The other part is easy and left to the reader. □ 

Theorem 3.10. An element f G Mod(N2g+i) is a Y -homeomorphism if 

and only if 

i) f ^T, f^ = te, where te is a Dehn twist about separating circle e. 

ii) C'Mod(N2g+i)(*e) contains two chains of Dehn twists Ei and Di, • • • , -D2g-i 
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such that all but Ei are about nonseparating circles, satisfying for all i, 
[tD„tEi] = 1 and Di G CMod(N2g+i)(/)- 

Hi) The chain Di, ■ ■ ■ ,-D2g-i is separating but does not separate a disc and 
the chain Ei cannot be chosen of bigger length. 

Proof. Let v be the tubular neighborhood of IJ Dj. Then x(^) = 2 — 2^/ and 
hence, 

x{N2g+i \v)=2-{2g+l)-{2- 2g) = -1. 

Also, V and hence N\v has two boundary components (the chain Di, • • • , -D2g-i 
has odd number of elements). Since it is also a separating chain we see that 
N2g+i = Ni^i U A''2,i or A^i^i U So, e must be the boundary of A''2,i 
or Si^i. The second case is not realizable by the condition (iii) because 
otherwise e would bound a one holed torus and we could replace the chain 
El with a longer chain Ei, E2 in the one holed torus. 

Since [/, Di] = 1, we can isotope / to fix Di pointwisely. Since [/, D2] = 1 
and f{Dir\D2) = Di nL'2 = pt, we can isotope / to fix D1UD2 pointwisely. 
Similarly, we can assume that / fixes DiU ■ ■ ■ L) D2g-i pointwisely. Hence, 
/ is supported outside of i^, because / cannot interchange the two sides of 
the tubular neighborhood. Moreover, since N2g+i \ = -/Vi,i U A''2,i and 
A^i^i does not support any nontrivial diffeomorphism which is identity on 
the boundary we see that / is supported in A''2,i. So, / is an element of 
Mod(N2,i) which is not a Dehn twist so that = t^, where e = dN2^i. In 
particular, Ei is the Dehn twist about the unique (up to isotopy) two-sided 
circle, say a, in the one holed Klein bottle bounded by e. Hence / is a 
Y-homeomorphism. 

The other direction is left to the reader. □ 

Remark 3.11. Using the notation of the above proof, let us continue our 
investigation of Y-homeomorphisms in the mapping class group of the Klein 
bottle with one boundary component. We have the presentation ([31 [7]) 

Mod(N2,i) = (y,ta I ytay-^ = t~^) = Z x Z, 

where ta is the Dehn twist Ei. One can easily see that 

C(Mod(N2,i)) = (tb) = Z, 

where t^ is y^. 

It is not hard to see that, any word in Z x Z has the form for 
m,n G Z, and this form is unique. Let x = y^t^. Then = y^t'^y'^t'^^. 
So, equals y'^^t^ if m is even, or y^*" if m is odd. Hence, any x = yf^ 
satisfies the equation = y'^, n G Z. Since t~^yt^ = yt^ up to conjugation 
y^ and y^ta are the only y-homeomorphisms supported inside a punctured 
Klein bottle. 

4. Outer Automorphisms of Mod(A''g) 
Theorem 4.1. Out(Mod(N2g+i)) injects into Z if g >2. 
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Proof. Consider the maximal tree of Dehn twists in Figure [3l 

It is well known that the Dehn twists ia^, i = ^, ■ ■ ■ ■,4:g — 2 and a Y- 
homeomorphism, say U, supported in the tubular neighborhood of the union 
of the circles 04^-4 and e (which is a copy of one holed Klein bottle) generate 
the mapping class group Mod(A''2g+i) (c.f. see (HJI^). 

Let $ : Mod(A''2g+i) Mod(A'^2g+i) be an automorphism. By the results 
of the previous section the image of this tree, which we denote by T, will be 
again a tree, where each maximal chain in T bounding a disc will be sent 
to a maximal chain in $(r) bounding a disc. Indeed, to get the maximal 
tree of circles corresponding to the maximal tree of Dehn twists 
i = 1, . . . jAg — 2, we can choose a hyperbolic metric on N2g-\-i, and then let 
bi be the unique geodesic in the homotopy class of a circle corresponding to 
the Dehn twist $(tai)- In particular, $(taj = h,, for each i (see also pLj). 

The tubular neighborhood of the tree, say v, of T is an orientable surface. 
Choose an orientation on v and on each Oj so that the intersections (a^ , Oj ) 
{i < j) is compatible with the orientation. Choose an orientation of a tubular 
neighborhood of the tree {61, • • • , 64^-2} and a diffeomorphism of ai onto hi. 
Keeping track of orientations we can extend this diffeomorphism to whole 
of T and hence to a neighborhood of T, call it again v. This diffeomorphism 
extends to a diffeomorphism of the surface since we need to extend through 
discs and a one holed real projective plane. 

Say, -0 : ^2g+i ^2g+i is one such diffeomorphism. Composing $ with 
V'iT^) we may assume that $ fixes all ta-. Let U' = ^{U), where U is as above. 
Since U commutes with all ta- for i < 4^ — 6, so is U'. Since a power of tc is 
a composition of ta-'s, i < 4g — 6, U' commutes with tc also (Figure H]). We 
can isotope U' to fix C pointwisely. Note that U' cannot interchange the 
two sides of a tubular neighborhood of C since the genus of -^''23+1 is odd. 
So, we may assume that U' is identity on a tubular neighborhood of C. So, 
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U' induces a diffeomorphism on -^2g+i which is the disjoint union of i 

and N-^^i. 




Figure 4. 

Since U' commutes with all ta/s, i < — 6, with tc, and the center of 
Mod(Sg_i^i) is (tc) we may assume that after an isotopy, U' is identity on 
Hence, it is supported on N-^^i. 

Now, we need to study Mod(A''3^i). Let us rename the circles a^g-zi, 
a4g_5, a4g_3 and 04^-2, and the Dehn twists about them by the letters Ai, 
A2, and B, respectively. It is known that Mod(A'^3^i) has the following 
presentation [5]: 




Figure 5. 

Generators:^!, A3, B and U. 
Relations: 

1) A1A3 = A^Ai 

2) AiBAi = BAiB, and A3BA3 = BA^B 

3) UAiU'^ = 

4) UBU-^ = A^^B-^A-i 

5) {A^Uf = = {AlA3Bf{= C). 

It turns out that replacing U with V = A^U gives a convenient set of 
generators and relations: 

Generators: Ai, A^,, B and V. 
Relations: 
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1') ^1^3 = ^3^1 

2') AiBAi = BAiB, and A3BA3 = BA3B 

3') VAiV-^ = A^^ 

4) VBV-^ = B-^ 

5') V^As = AsV^ 

6') = {AjA3Bf{= C). 

(Note that 5') follows from 6')). 

Since all these generators commute with tc, ^ maps Mod(A^3^i) into itself. 
Indeed, we have $(Ai) = Ai, $(^3) = ^3 and = B. 

To make things easier we will consider the subgroup of Mod(A''3^i) gener- 
ated by all Dehn twists. We will show that this subgroup, denoted by L, is 
a normal subgroup of index 2. 

Let W = V-^AsV which is y^gF"! = U-^A^U. Note that AiW' = 
W'Ai and W'V = VA^, for ah i. Also, B'^WB'^ = WB'^W and 
BW-^B = W-^BW-\ " Set L = (^1, ^3, B, W). It is easy to see that 
this is a normal subgroup of Mod(A?^3 1) generated by Dehn twists and hence 
V ^ L. Nevertheless, E L and hence L has index two in Mod(Ar3 1). So, 
L must be the subgroup generated by all Dehn twists. 

Now, any element of Mod(A''3^i) which is not a product of a Dehn twist 
has the form DV for some D ^ L. In particular, ^{V) = DV for some D. 
Since ^{V"^) = V^, {V^ = {AlA^Bf), we see that V'^ = {DVf = DVDV = 
DVDV-^V'^ = DD'Y"^, where D' = VDV'^. So, DD' = id. 

{D' is obtained from D by replacing Ai with A^^, B with B~^, W with 
A3 and As with W.) 

Apply <!> to V-^AiV = A^^ to get V-^D-^AiDV = = V'^AiV. It 
follows that AiD = DAi. Similarly, apply $ to V'^BV = B"^ to get BD = 
DB. So, D maps a tubular neighborhood of Ai and B diffeomorphically 
onto itself. By composing D with {AiB)^ = {BAi)^ if necessary we may 
assume that D is identity on a tubular neighborhood of A\ and B, where 
C4 is the boundary of the tubular neighborhood of the union of A\ and B. 

The complement of the tubular neighborhood of the union of A\ and B is 
a projective plane with two boundary components; let us call it Ni^2- Since 
Mod(Ari_2) = Z0Z = (C)0(C4), we get D = C'^AiBf"" for some m 
and n G Z. Now, use the relation VDV'^ = D'^ to get C'iAiB)-^'^ = 
C-"(^iB)-=^"^. It follows that n = 0. So, I? = (AiBf"". Hence, we asso- 
ciate an integer, namely m to each element of Out(Mod(N2g+i)). Indeed, 
this is a group homomorphism Out(Mod(N2g+i)) Z. To see this first note 
the following. Let $1 and $2 £ Out(Mod(N2g+i)), so that $1 and $2 are 
identity on all ta„ Ai, B and ^i{V) = {AiBf'^^V and ^2{V) = {AiBf'^^V. 
So, ($1 o ^2){V) = (^iS)3('"i+™2)y. 

To finish the proof of the theorem we need to show that such an auto- 
morphism (if it exist) cannot be an inner automorphism. To see that this is 
not an inner automorphism note the following: Any diffeomorphism which 
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commutes with the ah t^. is isotopic to the identity on the orientable com- 
ponent of N2g+i — C3. So, it must be isotopic to the identity on the whole 
surface N2g+i. However, the automorphism induced by the identity diffeo- 
morphism is the identity. So, it should send V to V also. It is easy to see 
in this case that the exponent m oi AiB must be zero. In particular, the 
homomorphism Out(Mod(N2g+i)) ^ Z is injective. 

□ 

Theorem 4.2. Out(Mod(N2g)) injects into Mod(i;o,4), for g>3. 

Proof. The idea of this proof is the same as the odd genus case. However, 
there are some technical differences. Consider the maximal tree of Dehn 
twists ta^, i = 1, ... jAg — 5, in Figure [U together which a y-homeomorphism 
supported in the one holed Klein bottle bounded by the circle e generate 
the mapping class group Mod(A''2g) (c.f. see [3l[7]). 




• «6 



"4s-9 



"4^-10 



4s-S 



-'4g-7 04^,^ 



Figure 6. 

The maximal tree above has tubular neighborhood diffeomorphic to 2(9-1) • 
So, if $ is an automorphism of Mod{N2g), let tb^ = ^{taj, i = 1, . . . , % — 5, 
then 5i, • • • , 643-5 form a tree as above. Similar to the odd genus we can 
find a diffeomorphism of a tubular neighborhood of the tree formed by Oj's 
(i = 1, . . . , 4^ — 5), call T onto that by 6j's, call T'. Each maximal subchain 
bounding a disc is sent to a maximal subchain bounding a disc. Therefore, 
we can find a diffeomorphism ipo : T [J {2{g - 2))D^ ^ T' U {2{g - 2))D^ 
which are both a copies of the orientable surface Sg_i_2- Then x{'^g-i,2) = 
2 - 2(5 - 1) - 2 = 2 - 2<7. So, x{N2g \ ^g-1,2) = (2 - 2g) - (2 - 2g) = 0. 
since, Sg_i^2 has two boundary components N \ Sg_i^2 must be a cylinder 
and the ends are glued to Sg_i^2 so that the surface becomes nonorientable. 
In particular, we see that N2g is obtained by gluing the two boundary com- 
ponents of T,g-i^2- Now, it is easy to extend the diffeomorphism ipo to N, 
say if. Note that by construction 'tpo{ai) = Oj, for all i = ,4g — 5, 
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where tpo is a diffeomorphism of orientable subsurfaces N2g \ 1^(04^-4) onto 
N2g\i'(hig-^A). So, the automorphism ■0* from Mod(A^2g) to Mod(A^2g) sends 
ta, to tb^, i = 1, . . . ,4^-5. 

Composing $ with ■07^ we may assume that ^{tai) = ta^ i = 1, ■ ■ ■ ,4:g — 5. 
Since ta^^_^ commutes with all other i^. it should be supported in the one 
holed Klein bottle obtained by cutting the surface N2g along a4g_8, a4g-7 
and a4g-6- However, there is a unique nontrivial two-sided circle inside this 
one holed Klein bottle, which is 043-4. So, ^{ta^g_^) = ia4g_4- 

Since $ fixes all ai, • • • , a^g-Q any diffeomorphism supported in N2^2 will 
be sent by $ to a diffeomorphism supported in N2^2- So, $ induces an 
automorphism of the mapping class group of the surface with boundary 
-^2,2 (Klein bottle with two boundary components), obtained by cutting the 
surface along the circles 04^-8 and a^g-Q. 

Let A'^4 denote the surface in Figure [71 which is obtained from N2^2 by 
gluing its boundary components. 

Note that any element of Mod(A'^2,2) yields an element of Mod(A^4). In- 
deed, the kernel of the natural homomorphism, Mod(A'^2,2) Mod(A'^4), is 
taig-gtaig-e (c-f- see Lemma 4.1. of [51). 

To go further we will need a presentation of the mapping class group 
Mod(A'4) given in Theorem 2.1. of [6j. 




Figure 7. 



We will rename some of the circles on N2g so that the presentation in [6] 
would not be confusing: The circles a^gs, 04^-7, 04^-5, ci4g-4 and the Dehn 
twists about them will be denoted Ai, A2, A3 and B, respectively. In [6j, it 
is given that the mapping class group of A^4 is generated by the Dehn twists 
Ai, A2, A3, B and a Y-homeomorphism called C/3. 

Lemma 4.3. The relation {U^B)"^ = ^ of Mod(A^4) gives 

{UsBf = {A[)'^-{A';)>^^ 

in Mod(A''2,2); for some ki and k2, where A'^ and A'( are the boundary com- 
ponents of Ni cut along Ai. Moreover in the mapping class group of N2g, 
we get {U3Bf = {ta,g_,f^{ta,,_,f^. 
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Proof. The relation {U^B)"^ = 1 holds in the mapping class group of 
(Theorem 2.1 (11) of [6j). Now Lemma 4.1 of [5] finishes the proof of the 
first statement. The second statement holds trivially. □ 

Let L be normal closure of ta-'s and tb. Hence, we have yio(i{N2g)/L = 
(U3) ~ Z2. Since ^ fixes all ta,. 's and tj,, ^ induces an automorphism ^ of 
Mod(iV23)/^ and so that ^{Us) = U3. So, ^(C/s) = U3X for some X £ L. 
Apply $ to U3A[ = A[U3 and UsA'l = A'lUs to get U^XA'^ = A[U3X = 
U3A[X. This implies that XA[ = A[X and similarly XA'I = A'(X. Also, 
U3A3 = A^^U3 yields U3XA3 = Aj^^f/sX = U3A3X. It follows that XA3 = 
A3X. So, X commutes with A[, A'( and A3. Indeed, by similar arguments 
X commutes with all ta^, i < 4:g — 5. 

Clearly, X cannot change the two sides of tubular neighborhoods of A'^ 
and A'( and hence X is identity on a neighborhood of A'^ and A". The 
mapping class X cannot change the two sides of a tubular neighborhood of 
A3 also, since otherwise we would have XA3 = A^^X. So, X is an element 
of the mapping class group of the four holed sphere component of the surface 
N2g cut along the curves A[ , A'( and A3 : 



A3 



A3 

Figure 8. 

Hence, X = {A[)'^^{A'()<{A's)''^{A'^)'^2w{B,C), for some integers ni , n[, n2, n'^ 
and word w{B,C), where A'^ and A'^ are the new boundary circles obtained 
by cutting the surface along ^43. 

This defines a map Out(Mod(N2g)) Mod(Eo,4), which is indeed a ho- 
momorphism by construction. It is clear that it is also injective. □ 

4.1. Some sporadic cases. In this section we will state some results about 
outer automorphism groups of some of the surfaces not covered in the above 
theorems. 

Proposition 4.4. 1) Mod(iV2) = Z2 x Z2 and hence Out(Mod(N2)) = S3, 
the symmetric group on three letters. 
2) Out(Mod(N2,i)) = Z2 X Z2. 
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3) Out(Mod(N3)) contains a copy 0/Z2. 

4) Out(Mod(N3j)) contains a copy of the infinite cyclic group 7L. 

Proof. The proof of (1) is immediate. One can prove part (2) directly using 
the presentation given in Remark 13. Hi 

To prove part (4) recah the presentation of the mapping class group of 
N^^i and the candidate automorphism defined at the end of Theorem 14.11 

Generators: Ai, A3, B and V. Relations: 
1') = 

2') AiBAi = BAiB, and A3BA3 = BA3B 
3') VAiV'^ = A^^ 
4) VBV-^ = B-^ 
5') V^A3 = A^V^ 

6') = {A\A3BY{= C), and the automorphism is given by ^{Ai) = Ai, 
$(^3) = ^3, <^>{B) = B and ^{V) = {AiBfV. It is easy to check that this 
map is really an automorphism of the mapping class group. The fact that 
any power of it is not an inner automorphism follows from the fact that any 
mapping class commuting with the Dehn twists Ai, A3 and B should be a 
power of the Dehn twist about C. However, C'^VC = V ^ {AiBf'^V for 
any m 7^ 0. This finishes the proof. 

Finally, almost the same argument given above for part (4) proves part 
(3). □ 
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